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This white paper outlines recommendations for minimum academic accountability criteria for schools in 
Los Angeles Unified School District (LAUSD), aligned with the Smarter Balanced Assessments (SBAC) and 
metrics from the state’s new California School Dashboard. This framework could apply to charter school 
renewal or replication decisions as well as potentially to non-charter school determinations of minimum 
performance standards (below which non-charter schools would be designated as chronically 
underperforming and potentially triggered for other interventions or mitigations). 

These recommendations draw on the California Charter School Association’s (CCSA’s) long history in 
using minimum academic accountability criteria to guide its advocacy for renewing and replicating 
charter schools - an approach that has been recognized locally and nationally. We have experience 
developing, testing, and implementing such a system and since 2010 we have used a two-stage 
accountability framework to guide our advocacy for California’s 1,200+ charter schools. 

Based on this experience, we recommend that LAUSD adopt a two-tier accountability system. This 
framework would first rely on a review of schools’ measurable, objective academic progress as measured 
by state assessments and college readiness indicators, using initial filters of status (proficiency), a 
comparison to schools serving similar demographics, and growth or college readiness. Second, for the 
relatively small number of schools that fail to meet any of the initial filters, the framework would utilize a 
“multiple measure review” by educational experts of other student outcome data, including on-site 
reviews. This mix between quantitative analysis and qualitative school review draws a balance between 
efficiency (given LAUSD’s 
large number of schools) 
and efficacy (by allowing 
educational experts to 
review the programs in 
practice). This new way 
forward learns from the 
past strengths of the API 
system and addresses 
some of its weaknesses. 

We recommend that the 
minimum criteria be 
academic-outcome 
focused. For charter 

schools, California state law requires increases in academic achievement for all student groups to be the 
most important factor in determining whether to grant a charter renewal [Ed. Code §§ 47607(a)(3)(A)]. 
We recognize there are instances where LAUSD might choose to deny the renewal or replication of a 
charter school based on fiscal, operational, or governance concerns. We recommend these be addressed 
on a case by case basis. 


CCSA's Approach to Accountability 

Benefits of CCSA’s 

Narrow range of clear, publicly available indicators 

Approach: 

of performance, academic achievement focused. 

+ Simple and predictable 


- • STATUS 

+ Allows fair, nuanced and 

ABOVE ANY 

• SIMILAR SCHOOLS 

deep review for subset of 

=SUPPORT 

• GROWTH (elem./middle) 

• COLLEGE READINESS (high) 

* 

schools that need it most 

+ Avoids arbitrary, unfortunate 
non-renewal decisions that 
lack sufficient nuance or 


BELOW ALL = SECOND LOOK 

consideration of unique 


(Deeper dive Multiple Measure Review) 

circumstances 
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Below we outline the specific recommended criteria (including the rationale behind each measure and a 
detailed description of the 2 nd tier “Multiple Measure Review”). The proposed metrics parallel portions 
of the charter renewal criteria currently in statue [Ed. Code §§ 47607(b)] requiring a charter school to 
meet at least one of several academic criteria before being renewed (specifically, a state rank or a similar 
schools rank of “4 or above” in 2 of the last 3 years). The paper presents modeling of how these 
academic accountability criteria could identify charter and non-charter schools within LAUSD. It also 
proposes that the LAUSD Board consider periodic revisions to ensure an increasing level of rigor over 
time, to adapt to a changing state accountability framework and to include newly released measures. 
The paper concludes with an appendix showing the alignment to state education code regarding charter 
renewal criteria. 

Why Not Just Use Metrics or “Colors” from the New State Dashboard? 

The state and local indicators in the California School Dashboard provide many new views into school 
performance, but do not lend themselves to a useful “minimum bar” for determining charter school 
renewal in their current format. Some of the measures do not represent student academic learning 
outcomes (such as suspension or chronic absenteeism) and the local indicators merely represent 
whether a school has completed a task (such as a survey), not a meaningful performance outcome. 
Moreover, the state has decided to avoid aggregating their metrics into a single measure view of 
performance. The State Board of Education, in its November 2016 meeting, in Item 3 on pages 3-4 of 
Attachment 3 . clarified that the Dashboard indicators (then referred to as evaluation rubrics) aren’t 
intended by statute for use in their current form as charter renewal criteria. 

The Dashboard criteria apply for identifying charter schools for continuous improvement intervention if 
“three or more student groups (or all the student groups if there are less than three student groups) had 
'not met’/'red' for one or more state or school priority identified in the charter for three out of four 
consecutive school years.” Given the numerous indicators and subgroups on each indicator, this presents 
a myriad of trip up points for charter schools, a high degree of complexity, and very little predictability. 

Moreover, high schools are particularly disadvantaged in the state’s new system. Under SBAC high 
schools only test in 11 th grade, and those tests are only included as a narrow subset of the College and 
Career Indicator (and only for students that graduate in 4 years). Without meaningful 9 th and 10 th grade 
state assessment data, a Multiple Measure Review process becomes even more critical for fairly and 
carefully evaluating high schools’ performance, particularly for high schools that keep students for a 5 th 
year or reengage credit deficient students. 

Many of the priority areas identified in the state’s dashboard align with CCSA’s Multiple Measure Review. 
Use of an aggregated set of clear academic metrics, combined with a nuanced review focusing on 
outcome measures bed to the charter’s listed priorities is a far more holistic and effective way to 
evaluate school performance for high stakes renewal decisions. 

Recommended Criteria 

Below we describe the specific metrics that could be used as initial filters and then additional metrics 
that could be used for deeper dive Multiple Measure Reviews. We note that these metrics should be 
adopted with an eye toward increasing a minimum bar over time, allowing for adaptation of the 
measures as the state releases improved data (such as academic growth or college/career readiness). It 
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is also crucial to develop early warning systems so schools know how they are doing prior to renewal and 
have the opportunity to prepare for a Multiple Measure Review if it seems likely that will be necessary. 

Schools would have to meet one of the following criteria to be above the initial filters of the Academic 
Accountability Framework and not require a Multiple Measure Review: 


1) STATUS MEASURE 

State Rank: 4 or above (in 2 of the last 3 yrs.) 

Above the bottom 5% on status measure (in 2 of the last 3 yrs.) (automate MMR if below) 


ABOVE ANY 
= SUPPORT 


2) GROWTH/POSTSECONDARY READINESS 

Elementary/ middle schools: Growth over time on SBAC exceeding the 75th percentile of growth 
statewide (e.g., using the “APD” measure, growing at least 18 scale score points on SBAC between 
2014-15 and 2016-17) 

High schools: 75% or more of graduates complete "a-g" requirements (in 2 of the last 3 yrs.) 


3) SIMILAR STUDENTS 

Similar Schools Rank: 4 or above (in 2 of the last 3 yrs.) 



BELOW ALL = MULTIPLE MEASURE REVIEW 


Initial Filters 

Status (Distance from Level 3/Average Point Difference): In 2014, CCSA developed a measure to help 
schools and parents better understand the Smarter Balanced Assessment (SBAC) scale scores and gauge 
improvement over time. The measure, now also used by the CDE and known as Distance From Level 3 
(DF3), compares a school's SBAC scale scores by grade to the state’s "met standard" (level 3). DF3 shows 
how the average student in the school performed on the SBAC, expressed as the number of scale score 
points above or below the “met standard." Whereas the state only reports this metric at the school and 
subject level (and only for schools serving 3-8 th grade students) with no aggregate for ELA and math and 
no ranking of schools’ performance relative to one another, CCSA takes the additional step to transform 
each subject into a singular statewide percentile ranking. We calculate each grade’s average point 
difference from level 3 and weight the average by each grade’s percent of test-takers. These weighted 
differences are added up to a schoolwide measure, and averaged across ELA and math to obtain a single 
status percentile per school. Then all schools are ranked to calculate a statewide percentile as well as 
deciles (CCSA refers to them State Ranks). Based on extensive modeling of which schools fall above and 
below various thresholds, CCSA recommends that the State Rank of 4 of performance statewide in two 
of the last three years should be the minimum threshold for this criterion. 

Similar Students Measure: Based on extensive research, CCSA has found that controlling for student 
demographics shows a more accurate picture of how well a school is performing with the student 
population it serves than relying on a school’s average test results alone. This type of measure has 
historically been included in the state’s charter renewal criteria and ensures schools aren’t penalized for 
intentionally locating schools in neighborhoods with the most historically disadvantaged student 
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populations. Using the state of California’s prior School Characteristics Index as a guide, CCSA has 
calculated a Similar Students Measure for all schools in the state since 2011. This measure controls for 
student demographics and compares the predicted to actual performance based on statewide outcomes 
for key subgroups. The measure has been vetted by experts and was demonstrated to be robust ( see 
methodology and technical guide sections of this CCSA webpage) . Similar to the transformation of our 
status criterion, CCSA has also ranked all schools statewide on this metric to calculate deciles (CCSA 
refers to them Similar Student Ranks). CCSA recommends that performing at or above the Similar 
Student Rank of 4 in two of the last three years would be the minimum threshold for this criterion. 

Average Point Difference Growth (Change) for elementary and middle schools: Until the state of 
California issues a SBAC-based longitudinal student growth measure, CCSA has calculated an 
improvement measure for elementary and middle schools, comparing improvement from prior to 
current year scale scores. For elementary and middle schools, CCSA compares a school’s increase in scale 
score points from one year’s status measure to the next. 1 A school with more of its students making 
progress toward the “met standard’ compared to the previous year would be given credit on our 
framework. In fall 2016, when the state only had two years of operational SBAC data, CCSA’s framework 
established that any elementary or middle school that rated in the 75 th percentile of students (or what 
translated to at least 15 points of scale score growth) on the Distance from Level 3/Average Point 
Difference Measure from 2015 to 2016 would meet this criteria. This threshold is intended to be 
relatively high compared to the other filters, with the expectation that lower performing schools should 
show larger positive gains with their students over time in order to receive renewal support. Now that 
there have been three testing periods, the 75 th percentile cut point can be used to target achievement 
across all three testing periods, which roughly translates to 18 points of growth between 2015 and 2017. 
CCSA recommends performing at or about the 75 th percentile in growth statewide across the last 3 years 

as a minimum threshold for this criterion. 

College Readiness: Percent of 12 th Graders meeting “a-g” criteria: For high schools, we do not include a 
year-over-year improvement measure in our initial filters, since only 11 th graders are tested and 
represent a different group of students each year. Instead, CCSA has utilized a measure of postsecondary 
readiness, specifically the "a-g" requirements (college-preparatory coursework students take in high 
school that prepares them to apply to a CSU or UC school). These courses must be approved by the 
universities as being academically challenging and providing students with a breadth of knowledge and 
analytical thinking skills that demonstrate preparation for and indicate likely success in college. CCSA 
currently recommends that 75% or more of 12th grade graduates completing all "a-g" requirements in 

two of the last three years would be the minimum threshold for this criterion . Schools for whom college 
preparation is not a primary focus will have the opportunity to show other measures of postsecondary 
readiness and success in the Multiple Measure Review (such as career preparation, internships, job 
placements, certifications and other measures). As noted in the section below on proposed 
improvements over time, once the state has finalized a College and Career Indicator with both status and 
growth components, it might make sense to transition to this more holistic measure of college and 
career readiness. 


1 Note that while we do calculate a growth score for high schools, we do not include this measure in our initial 
filters, since only 11 th graders are tested and thus are unable to demonstrate year-over-year growth on this 
measure with the same students. Instead, to measure graduates’ level of postsecondary preparedness, CCSA uses 
schools’ a-g rate. 
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Bottom 5%: To ensure that schools with very low levels of proficiency were reviewed with additional 
scrutiny, CCSA requires any school in the bottom 5% in the last two of three years on Distance from Level 
3/Average Point Difference percentiles to undergo the Multiple Measure Review process, regardless of 
whether the school is above any of CCSA’s other initial filters. 

Minimum Threshold and Exclusions: Schools that are above ANY of the initial filters and not in the 
bottom 5% would meet the minimum thresholds of this accountability framework, and would not be 
highlighted for additional review. We would recommend that these schools receive a positive renewal 
recommendation, except for a small subset of schools facing non-renewal for non-academic reasons as 
outlined in the non-academic accountability section below. Schools below ALL of the initial filters would 
undergo a more holistic review of multiple measures as a second ber to the accountability framework. 
CCSA recommends excluding from this framework any school that is designated by the state as 
Alternative or that has fewer than 30 valid test scores on the SBAC. Also note that at this time we are 
not making any additional recommendations regarding alternative and small school accountability but 
are monitoring the actions of the state regarding these categories of school accountability. 

Multiple Measure Review (MMR) 


For the subset of schools that are below all the initial filters or are in the bottom 5% of performance on 
the status measure, CCSA recommends a Multiple Measure Review (MMR). We understand that some 
schools may have circumstances that require a deeper analysis and review of their internal non-public 
test data to determine and demonstrate that students are making academic progress. We have 
conducted this process within CCSA by utilizing a cross-section of researchers and school leaders to 
ensure a robust, nuanced, and fair review. For LAUSD, we recommend that MMR Committees be 
comprised of a combination of a staff member from the LAUSD Superintendent’s Office, a member of an 
independent organization such as CCSA, and a neutral third party. 

Using a template based on the academic LCAP priorities, evaluators would review a much larger set of 
data than just test scores and a-g rates. Multiple Measure Review templates would have a heading for 
each LCAP priority, and prior to the Review the template would be populated by any publicly available 
information on the California School Dashboard. Under each heading, relevant data could be included 
based on school submissions. It is important to note that the MMR would focus on student outcome 
measures that are relevant to the particular school’s charter petition, stated mission, and LCAP goals. As 
examples, this review could include indicators such as: 


EL Reclassification rates 
EL Language proficiency rates 
Special Education Exits 
Graduation Rates 
Dropout Rates 

Suspension and Expulsion rates 
AP/SAT/ACT Rates 


a-g completion 
College Acceptance Rates 
College going rates 

Norm-referenced internal assessments 
monitoring in-year learning growth. 
Comparison school performance 
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CCSA has provided schools with this linked guiding document that illustrates examples of student 
outcome data aligned with a school’s unique mission that schools may choose to share. It is not meant to 
be an exhaustive list but provides schools guidance on providing outcome based data as part of their 
MMR submission. Schools would be given a window of time to submit data related to each of eight state 
LCAP identified priorities including student and parent engagement/satisfaction, course offering variety, 
and internal assessments results. Evaluators would conduct an initial group review of the collected data 
to determine whether the school has been academically successful in ways that are not easily accessible 
via the initial filters alone. The reviewers would compare the school’s stated priorities as identified in its 
charter petition and LCAP with its outcomes. For example, a school may seek to serve a high proportion 
of students that enter the school below grade level and as a result have low standardized test scores. 

This same school uses internal norm referenced Common Core aligned assessment data to show 
students come in to the school significantly below grade level, yet the students make more than one 
year of grade level growth for one year in the school. Another school could be focused on preparing 
students for post-secondary success and have implemented structures to track college acceptance and 
enrollment rates. 

With each school’s LCAP as a guide, evaluators would understand the school’s progress with its students 
in terms of the school leaders’ reasoning for starting a charter. Schools that can demonstrate the value 
add of their school by providing compelling data that tells a different story of student academic 
achievement than publicly available measures alone would be categorized as “above minimum criteria as 
a result of the Multiple Measure Review” and would be recommended for renewal. 

Principles for Evaluating School Success in the Multiple Measure Review include the following: 

1) We expect charters to be accountable for the achievement growth of their students for the term 
of their existing charter. A gauge is to identify where students were academically when they 
entered the school, was their academic progress monitored and are they making a year or more 
(pending on how far behind students come in) for their time in the school? What is the 
evidence? 

2) Students should be set up for success in high school and should have access to the post¬ 
secondary outcomes of their choice and data should indicate that they will have the chance to be 
successful when they get there. For high schools, what are the post-secondary outcomes for high 
school students including what indicators show that students will be successful when they get to 
their post-secondary option of choice? What is the evidence? For K-8, does the data indicate 
students will be prepared for success in high school? What is the evidence? 

3) Are there unforeseeable events beyond the control of the school that impacted the school’s 
ability to accomplish greater outcomes for students and have these issues been rectified or will 
they likely persist into the next charter term ? What is the evidence? 

4) Is there stability and confidence in leadership moving forward that they can continue the upward 
growth patterns or significantly reshape the existing outcomes from where they are? What is the 
evidence? 

Evaluators would conduct a school site visit in the year prior to renewal to would conduct a school site 
visit to further inform their assessment about the success or failures of the educational offerings. The 
Review Committee could identify clear standards by which schools would be evaluated based on 
education best practice. Reviewers would observe a significant number of classrooms during 
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instructional time and assess the engagement level of students with rigorous content. Ultimately, these 
educational experts could offer their opinion of the efficacy of the educational program, provide a rating 
of the programs, and a recommendation to the charter school’s authorizes With a more thorough review 
process, those schools with an education delivery system that focuses on specialized course offerings, or 
academic themes such as art or music, could demonstrate whether their students are successful on 
alternative academic measures in addition to the Math and ELA SBAC tests. 

CCSA has also implemented an early warning site visit prior to the Multiple Measure Review. Charter 
schools below initial filters but not in their year of charter renewal would receive outreach to discuss 
their upcoming renewal plans and would be offered a site visit that includes an instructional walk 
through and discussion with the school leader to support a path to higher success for the charter school 
prior to their year of renewal. Schools that increase their performance such that they are above any one 
of the initial filters in their year of renewal (where they were previously below) do not have to 
participate in a Multiple Measure Review. The initial filters along with the Multiple Measure Review 
system, including a site visit for those schools below all initial filters, would allow LAUSD to focus its 
limited resources on those schools that have a demonstrated need for intervention. 

Recommended changes over time 

Given that the state will continue evolving its accountability framework over the next few years, we 
recommend creating an accountability framework with an intent to evolve it over time to ensure rising 
levels of rigor and adaptation and alignment to the state where reasonable. 

In order to ensure that students are provided access to the highest quality student learning in LAUSD, 
the accountability criteria should be set with the understanding that the minimum floor will rise over 
time. For example, the comparison to similar schools or the minimum preparedness expectation in 
college and career readiness could evolve into more rigorous bars over time. What is important is 
establishing a framework that is flexible to respond to increases in statewide and regional academic 
performance with an express commitment from day one to agree to ever-increasing levels of rigor to 
ensure students have the highest quality schooling options within LAUSD. 

The state is in the process of developing and releasing new measures which may prove more effective at 
measuring school performance than the current measures used as initial filters. For example, the state’s 
College and Career Readiness Indicator is more holistic than just a-g completion rates. Once this measure 
is in place with a multi-year track record, more current years of data, and both status and change 
elements, it would be appropriate to consider using it instead of the current a-g measure. Also, the state 
has indicated its intent to release a measure of student growth that would more effectively measure 
growth instead of the current aggregated measure of school improvement on SBAC from one year to the 
next. These measures, once finalized, may prove to be effective initial filters for narrowing the group of 
schools requiring Multiple Measure Reviews. 

As new measures or adaptations to the model are considered, it is important to test and model their 
impact to ensure no demographic of students or model of schools is unfairly penalized by the new 
metrics. This has been a crucial step in CCSA’s testing of its framework each time it has made a change or 
adaptation to its measures. 

Specific Modeling of LAUSD Charter and Non-Charter Schools 
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Initial Filters 

In this section, we describe how CCSA’s accountability framework could be applied to schools in LAUSD. 

In 2016-17 183 autonomous charter schools were at least 4 years old, not alternative, or serving very 
small groups of students (less than 30 valid SBAC scores). As Figure la shows that of these 183 
autonomous charter schools, 10% (19 charter schools) were below CCSA’s initial filters based on 2016-17 
data. If applied to non-charters, 28% (175 schools) would fall below CCSA’s initial filters. Every year 
around 20% of charter schools undergo renewal, so we project that in a typical year, one fifth of these 19 
charters (anywhere from 4-5 charter schools depending on the year) might undergo an extensive 
Multiple Measure Review process. If non-charters were included in the process on a similar timeline, 33- 
37 might undergo the Multiple Measure Review. 


Figure la: Demographics for LAUSD and Schools Below Initial Filters 2 



LAUSD Schools Below CCSA Initial 
Filters 

LAUSD Schools Above Initial 
Filters 

LAUSD Average (Above & Below 
combined) 

Average 

Charter 

Non- 

Charter 

Affiliated 

Charters 

Charter 

Non- 

Charter 

Affiliated 

Charters 

Charter 

Non- 

Charter 

Affiliated 

Charters 

% of school type in LAUSD 

10% 

28% 

2% 

90% 

72% 

98% 

100% 

100% 

100% 

School Count 

19 

175 

1 

164 

443 

53 

183 

618 

54 

Estimated # of MMR Schools 

4 

35 

0 

- 

- 

- 

- 

- 

- 

% Below DF3/APD Rank 

100% 

100% 

100% 

23% 

34% 

0% 

31% 

53% 

2% 

% Below 5th 

68% 

34% 

0% 

0% 

0% 

0% 

7% 

10% 

0% 

% Below Growth 

79% 

91% 

100% 

70% 

63% 

66% 

71% 

71% 

67% 

% Below Similar Schools Rank 

74% 

93% 

100% 

8% 

12% 

15% 

15% 

35% 

17% 

% Below a-g rate 

67% 

100% 

- 

26% 

97% 

100% 

32% 

98% 

100% 


As indicated in Figure lb, charters and non-charter schools vary across years in averages on our metrics, 
with more movement among charters. For charters below CCSA’s initial filters, schools averaged a 1.3 
State Rank for the past two years whereas non-charters averaged .4 higher with a 1.7 State Rank. 
Conversely, charters above and below CCSA’s initial filters ranked higher on average than non-charter 
schools on Similar Student Ranks for the past three years. 

Figure lb: Academic Averages for Schools Below Initial Filters (vs. LAUSD & State Overall) 



LAUSD Schools Below CCSA 
Initial Filters 

LAUSD Schools Above Initial 
Filters 

LAUSD Average (Above & 
Below combined) 

Average 

Charter 

Non- 

Charter 

Affiliated 

Charters 

Charter 

Non- 

Charter 

Affiliated 

Charters 

Charter 

Non- 

Charter 

Affiliated 

Charters 

(DF3/APD) State Rank 2017 

1.3 

1.7 

4.0 

5.5 

4.8 

8.2 

5.1 

3.9 

8.1 

(DF3/APD) State Rank 2016 

1.3 

1.7 

3.0 

5.6 

4.8 

8.2 

5.2 

3.9 

8.1 

(DF3/APD) State Rank 2015 

2.0 

1.7 

3.0 

5.3 

4.4 

7.9 

4.9 

3.6 

7.8 

Similar Schools Rank 2017 

2.9 

2.5 

2.0 

7.2 

6.4 

7.2 

6.7 

5.3 

7.1 


2 Figures la, lb, 2 and 3are analyses conducted on schools that have been opened longer than 4 years. 
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Similar Schools Rank 2016 

3.0 

2.3 

1.0 

7.2 

6.0 

6.3 

6.8 

5.0 

6.2 

Similar Schools Rank 2015 

4.3 

2.7 

1.0 

7.1 

5.8 

5.5 

6.8 

4.9 

5.4 

3-year growth 

-4.2 

3.7 

17.9 

8.5 

13.3 

13.4 

7.2 

10.6 

13.5 

a-g Rate 16 

64% 

51% 

- 

86% 

56% 

53% 

83% 

55% 

53% 

a-g Rate 15 

54% 

48% 

- 

86% 

48% 

47% 

82% 

48% 

47% 

a-g Rate 14 

50% 

41% 

- 

86% 

43% 

41% 

81% 

42% 

41% 


As CCSA developed its recommended criteria, we carefully assessed how schools serving different 
demographic groups would be impacted by these criteria. While the Similar Students Rank plays a critical 
role in ensuring schools are not penalized by serving differing groups of students, as shown in Figure 2 
below, there are demographic differences when comparing schools above and below the criteria. The 
Multiple Measure Reviews thus take on even greater importance in fully and deeply assessing how these 
different student groups are being served in these schools. 

Figure 2: LAUSD All Schools & Initial Filters 


School Type & Demographics 

Initial Filters (Below 2 of 3 years) 

Status SSR Growth/AG Bottom 5th 

Below All 

Filters 

Above Filters 

# of schools identified 

383 

251 

627 

73 

195 

660 

% also below all other filters 

51% 

71% 

29% 

100% 

- 

- 

Elementary 

58% 

66% 

62% 

29% 

57% 

66% 

Middle 

25% 

19% 

18% 

42% 

25% 

15% 

High School 

17% 

15% 

20% 

29% 

18% 

19% 

Latino 

84% 

81% 

75% 

77% 

83% 

72% 

Asian 

1% 

2% 

4% 

1% 

1% 

4% 

Black 

10% 

8% 

8% 

19% 

10% 

7% 

White 

3% 

6% 

10% 

2% 

3% 

12% 

Other 

2% 

3% 

4% 

2% 

2% 

4% 

English Language Learners 

32% 

31% 

25% 

30% 

32% 

23% 

FRL 

90% 

86% 

81% 

90% 

89% 

78% 

SPED 

13% 

12% 

12% 

14% 

13% 

11% 


As Figure 2 shows, a little less than half of all schools in LAUSD are identified as below CCSA’s Status, 
while almost three-quarters are below on growth/a-g initial filters. A third of schools are below SSR 
which indicates that by accounting for these schools’ demographics in the SSR, it’s evident that the vast 
majority of LAUSD schools have levels of achievement that are similar or greater to schools across the 
state of California serving comparable student demographics. Figure 2 also shows that the grades served 
and demographics of the schools below CCSA’s initial filters are relatively similar to schools that are 
above our filters - meaning that the initial filters do not penalize any one demographic subgroup or 
grade level. Despite being generally similar, there are some differences between schools above and 
below CCSA’s initial filters. Compared to schools above CCSA’s initial filters, middle schools are slightly 
more likely to be considered under CCSA’s Status and Similar Students Measure filters. Middle schools 
are also much more likely to be considered automatically below the 5 th percentile, with 46% falling below 
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this filter. Schools below filters are demographically similar, though there are some exceptions. Schools 
below on Status have a larger Latino population (84% of student population) compared to schools that 
are above CCSA’s initial filters (72%), as well as more English Learners (32% vs 23%) and Free or Reduced- 
Price Lunch students (90% vs 78%). African American students are overrepresented at schools that are in 
the bottom 5% on APD (19%) compared to the schools above CCSA’s filters (7%), as are Free-Reduced 
Price Lunch students (90% vs 78%). Schools automatically identified for the MMR because they’re in the 
bottom 5% on SBAC test results are least like the schools above CCSA’s initial filters. 

Nineteen charter schools and 175 non-charter schools are identified as performing below ALL initial 
filters of CCSA’s accountability framework (and thus would necessitate a Multiple Measure Review 
before receiving a “support” recommendation). Other helpful metrics of performance not specifically 
included in our recommended framework but often used to measure school performance include four- 
year cohort graduation rate, percentage met/exceeded on the SBAC tests, and percentage of students 
continuously enrolled from fall to spring. Below are statistics representing the average performance of 
charters and non-charters in LAUSD that are below CCSA’s initial filters: 


Charter Schools Below Initial Filters 

• 10% of charters (19 schools) 

• Average State Rank of 1 (out of 10) 

• 64% a-g rate 

• Average Similar Schools Rank of 3 (out 
of 10) 


Non-Charter Schools Below Initial Filters 

• 28% of non-charters (175 schools) 

• Average State Rank of 2 (out of 10) 

• 51% a-g rate 

• Average Similar Schools Rank of 3 (out 
of 10) 


CCSA recommends using only academic indicators from the CA School Dashboard in the accountability 
system’s initial filters. An accountability system that includes many different filters is complicated, 
making it harder for school leaders to understand the accountability process and more likely that 
otherwise high performing, successful schools are getting tripped up/threatened with non-renewal over 
one subgroup or non-academic indicator. Moreover, schools should be held accountable for the primary 
focus of their efforts: student outcomes. Academic-based initial filters are not meant to be an exhaustive 
study of schools’ progress with students, but a transparent and straightforward way of determining 
which charter schools deserve a closer and more nuanced look. Other school information, including the 
non-academic information provided on the new California School Dashboard, can be studied after the 
initial filters have identified the subset of at-risk schools. 


For comparison purposes, Figure 3 below shows that schools’ performance across the California School 
Dashboard is well correlated with the specific academic accountability criteria we propose. We see that 
schools identified as being below the initial filters of the accountability framework also are more likely to 
have higher suspension rates and less college and career preparedness. This table provides a greater 
level of confidence that a small subset of academic-focused initial filters can identify the right subset of 
schools for Multiple Measure Reviews without needing to look at numerous, complex and confusing 
combinations of colors, subgroups, and indicators on the state dashboard. 

Figure 3: LAUSD Schools & Initial Filters 
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School Type 

Suspension 
Rate (2016-17) 

CCI Prepared 
(2015-16) 

English Learner 
Progress (2015-16) 

Chronic 

Absenteeism 

(2015-16) 

Graduation 
Rate (2015-16) 

Below Criteria in LAUSD 

Autonomous Charter 

4% 

22% 

77% 

22% 

78% 

Non-Charter 

1% 

31% 

72% 

24% 

76% 

LAUSD Average 

Autonomous Charter 

2% 

39% 

77% 

12% 

90% 

Non-Charter 

1% 

34% 

74% 

16% 

83% 

State Average (All Schools) 

4% 

36% 

76% 

15% 

88% 


An accountability framework with initial filters based on student-outcomes has the advantage of being 
clear and easy for school leaders to understand, while guaranteeing that the most important result of 
schooling is being assessed. At the same time, because of the strong association between student 
outcomes and other non-academic indicators, the lowest performing schools on other LCAP indicators 
will also be identified. 

Summary 

The recommendations proposed in this white paper outline a clear path forward to ensure charter 
school renewals continue to prioritize increases in academic achievement for all student groups, in 
keeping with state law. The proposed framework parallels current charter renewal criteria (state rank 
and similar schools rank indicators) referred to in state statute. The two-tier Academic Accountability 
Framework minimizes complexity and trip up points for schools, aligns to California’s eight state priorities 
under LCFF, accounts for individual school’s unique missions and priorities and represents a holistic and 
effective way to evaluate school performance for high stakes renewal decisions, such as those taking 
place in Los Angeles. Academic accountability is a key component to protecting the autonomy and 
flexibility afforded to charter schools that allows them to help students succeed and thrive. It is also 
critically important to the CCSA, which has a longstanding commitment to ensuring California charter 
schools help their students meet high standards of academic excellence. Leveraging the 
recommendations in this white paper, LAUSD can create a framework and renewal process that ensures 
students attend high performing schools of diverse missions and types, that meet their unique needs 
and increase their student learning and long term success. 
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APPENDIX: Relevant State Education Code Governing Charter Renewals 

As California’s Academic Performance Index has been discontinued and as the California State Board of 
Education (SBE) revises the state’s accountability system, there have been questions about which criteria 
should apply to charter school renewal decisions. The SBE has sought clarity from its legal counsel about 
which standard to use given the absence of API scores and rankings and the intent to not recreate an API 
in the future, noting “the purpose of a performance index no longer aligns with the state’s expectations 
to adopt a multiple measures accountability system that supports continuous improvement.” 3 While 
SBE’s legal counsel has assured the State Board that they may continue to use both statutes, there has 
been considerable debate among SBE Members, law firms representing charter schools and local 
authorizers. The SBE has indicated it would like to seek clarity-either through legislation or guidance to 
authorizers-on which criteria should be used moving forward. 

Below are overviews of the existing renewal statutes. 

California Education Code §§ 47605 and 47606.5 require charter schools to be held accountable to the 
state priorities applicable to the program and grades served by the charter and the goals stated in the 
school’s charter as approved by its charter authorizes The State Board has also acknowledged these 
sections in applying the LCFF dashboard to charter schools. It has also acknowledged that the rubric 
criteria triggering intervention in Education Code §§ 47607.3 (explicit criteria for charter schools to be 
identified for intervention under LCFF, which, if unsuccessful could lead to the revocation of the school’s 
charter) are different from the charter renewal criteria (see Item 3. Attachment 3 of the November 2016 
State Board of Education meeting ). 


Education Code Section 52064.5. 

(a) On or before October 1, 2016, the state board shall adopt evaluation rubrics for all of the following 
purposes: 

(1) To assist a school district, county office of education, or charter school in evaluating its strengths, 
weaknesses, and areas that require improvement. 

(2) To assist a county superintendent of schools in identifying school districts and charter schools in need 
of technical assistance pursuant to Section 52071 or 47607.3, as applicable, and the specific priorities 
upon which the technical assistance should be focused. 

(3) To assist the Superintendent in identifying school districts for which intervention pursuant to Section 
52072 is warranted. 

(b) The evaluation rubrics shall reflect a holistic, multidimensional assessment of school district and 
individual schoolsite performance and shall include all of the state priorities described in subdivision (d) 
of Section 52060. 


3 California State Board of Education, September 2015 Agenda, Item 14, Attachment 1: Proposed Framework for 
the New Accountability System, page 8 (dated 8/21/15). 
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(c) As part of the evaluation rubrics, the state board shall adopt standards for school district and 
individual schoolsite performance and expectations for improvement in regard to each of the state 
priorities described in subdivision (d) of Section 52060. 


Education Code Section 47607. 

(a) (1) A charter may be granted pursuant to Sections 47605, 47605.5, and 47606 for a period not to 
exceed five years. A charter granted by a school district governing board, a county board of education, or 
the state board may be granted one or more subsequent renewals by that entity. Each renewal shall be 
for a period of five years. A material revision of the provisions of a charter petition may be made only 
with the approval of the authority that granted the charter. The authority that granted the charter may 
inspect or observe any part of the charter school at any time. 


(2) Renewals and material revisions of charters are governed by the standards and criteria in Section 
47605, and shall include, but not be limited to; a reasonably comprehensive description of any new 
requirement of charter schools enacted into law after the charter was originally granted or last renewed. 


(3) 

(A) The authority that granted the charter shall consider increases in pupil academic achievement for 

all groups of pupils served by the charter school as the most important factor in determining whether to 

grant a charter renewal. 



(B) For purposes of this section, “all groups of pupils served by the charter school” means a numerically 
significant pupil subgroup, as defined by paragraph (3) of subdivision (a) of Section 52052, served by the 
charter school. 


(b) Commencing on January 1, 2005, or after a charter school has been in operation for four years, 
whichever date occurs later, a charter school shall meet at least one of the following criteria before 
receiving a charter renewal pursuant to paragraph (1) of subdivision (a): 


(1) Attained its Academic Performance Index (API) growth target in the prior year or in two of the last 


three years both schoolwide and for all groups of pupils served by the charter school. 



school is at least equal to the academic performance of the public schools that the charter school pupils 


would otherwise have been required to attend, as well as the academic performance of the schools in 


the school district in which the charter school is located, taking into account the composition of the pupil 


population that is served at the charter school. 


(B) The determination made pursuant to this paragraph shall be based upon all of the following: 


(i) Documented and clear and convincing data. 
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(ii) Pupil achievement data from assessments, including, but not limited to, the Standardized Testing and 
Reporting Program established by Article 4 (commencing with Section 60640) of Chapter 5 of Part 33 for 
demographically similar pupil populations in the comparison schools. 

(iii) Information submitted by the charter school. 

(C) A chartering authority shall submit to the Superintendent copies of supporting documentation and a 
written summary of the basis for any determination made pursuant to this paragraph. The 
Superintendent shall review the materials and make recommendations to the chartering authority based 
on that review. The review may be the basis for a recommendation made pursuant to Section 47604.5. 

(D) A charter renewal may not be granted to a charter school prior to 30 days after that charter school 
submits materials pursuant to this paragraph. 

(5) Qualified for an alternative accountability system pursuant to subdivision (h) of Section 52052. 

(c) (1) A charter may be revoked by the authority that granted the charter under this chapter if the 
authority finds, through a showing of substantial evidence, that the charter school did any of the 
following: 

(A) Committed a material violation of any of the conditions, standards, or procedures set forth in the 
charter. 

(B) Failed to meet or pursue any of the pupil outcomes identified in the charter. 

(C) Failed to meet generally accepted accounting principles, or engaged in fiscal mismanagement. 

(D) Violated any provision of law. 

(2) The authority that granted the charter shall consider increases in pupil academic achievement for all 
groups of pupils served by the charter school as the most important factor in determining whether to 
revoke a charter. 

(d) Before revocation, the authority that granted the charter shall notify the charter school of any 
violation of this section and give the school a reasonable opportunity to remedy the violation, unless the 
authority determines, in writing, that the violation constitutes a severe and imminent threat to the 
health or safety of the pupils. 

(e) Before revoking a charter for failure to remedy a violation pursuant to subdivision (d), and after 
expiration of the school’s reasonable opportunity to remedy without successfully remedying the 
violation, the chartering authority shall provide a written notice of intent to revoke and notice of facts in 
support of revocation to the charter school. No later than 30 days after providing the notice of intent to 
revoke a charter, the chartering authority shall hold a public hearing, in the normal course of business, 
on the issue of whether evidence exists to revoke the charter. No later than 30 days after the public 
hearing, the chartering authority shall issue a final decision to revoke or decline to revoke the charter, 
unless the chartering authority and the charter school agree to extend the issuance of the decision by an 
additional 30 days. The chartering authority shall not revoke a charter, unless it makes written factual 
findings supported by substantial evidence, specific to the charter school, that support its findings. 
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(f) (1) If a school district is the chartering authority and it revokes a charter pursuant to this section, the 
charter school may appeal the revocation to the county board of education within 30 days following the 
final decision of the chartering authority. 

(2) The county board of education may reverse the revocation decision if the county board of education 
determines that the findings made by the chartering authority under subdivision (e) are not supported 
by substantial evidence. The school district may appeal the reversal to the state board. 

(3) If the county board of education does not issue a decision on the appeal within 90 days of receipt, or 
the county board of education upholds the revocation, the charter school may appeal the revocation to 
the state board. 

(4) The state board may reverse the revocation decision if the state board determines that the findings 
made by the chartering authority under subdivision (e) are not supported by substantial evidence. The 
state board may uphold the revocation decision of the school district if the state board determines that 
the findings made by the chartering authority under subdivision (e) are supported by substantial 
evidence. 

(g) (1) If a county office of education is the chartering authority and the county board of education 
revokes a charter pursuant to this section, the charter school may appeal the revocation to the state 
board within 30 days following the decision of the chartering authority. 

(2) The state board may reverse the revocation decision if the state board determines that the findings 
made by the chartering authority under subdivision (e) are not supported by substantial evidence. 

(h) If the revocation decision of the chartering authority is reversed on appeal, the agency that granted 
the charter shall continue to be regarded as the chartering authority. 

(i) During the pendency of an appeal filed under this section, a charter school, whose revocation 
proceedings are based on subparagraph (A) or (B) of paragraph (1) of subdivision (c), shall continue to 
qualify as a charter school for funding and for all other purposes of this part, and may continue to hold 
all existing grants, resources, and facilities, in order to ensure that the education of pupils enrolled in the 
school is not disrupted. 

(j) Immediately following the decision of a county board of education to reverse a decision of a school 
district to revoke a charter, the following shall apply: 

(1) The charter school shall qualify as a charter school for funding and for all other purposes of this part. 

(2) The charter school may continue to hold all existing grants, resources, and facilities. 

(3) Any funding, grants, resources, and facilities that had been withheld from the charter school, or that 
the charter school had otherwise been deprived of use, as a result of the revocation of the charter shall 
be immediately reinstated or returned. 

(k) A final decision of a revocation or appeal of a revocation pursuant to subdivision (c) shall be reported 
to the chartering authority, the county board of education, and the department. 


Ed. Code §§ 52052(e)(4): 
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(4) Any school or school district that does not receive an API calculated pursuant to subparagraph (F) of 

paragraph (2) shall not receive an API growth target pursuant to subdivision (c). Schools and school 
districts that do not have an API calculated pursuant to subparagraph (F) of paragraph (2) shall use o 

ne 

of the following: 

(A) The most r 

(B) An average 

ecent API calculation, 
of the three most re( 

:ent annual API calculations. 


(C) Alternative measures that show increases in pupil academic achievement for all groups of pupils 

schoolwide and among significant subgroups. 



Education Code Section 47607.3. 

(a) If a charter school fails to improve outcomes for three or more pupil subgroups identified pursuant to 
Section 52052, or, if the charter school has less than three pupil subgroups, all of the charter school’s 
pupil subgroups, in regard to one or more state or school priority identified in the charter pursuant to 
subparagraph (A) of paragraph (5) of subdivision (b) of Section 47605 or subparagraph (A) of paragraph 
(5) of subdivision (b) of Section 47605.6, in three out of four consecutive school years, all of the 
following shall apply: 

(1) Using an evaluation rubric adopted by the state board pursuant to Section 52064.5, the chartering 
authority shall provide technical assistance to the charter school. 

(2) The Superintendent may assign, at the request of the chartering authority and with the approval of 
the state board, the California Collaborative for Educational Excellence to provide advice and assistance 
to the charter school pursuant to Section 52074. 

(b) A chartering authority shall consider for revocation any charter school to which the California 
Collaborative for Educational Excellence has provided advice and assistance pursuant to subdivision (a) 
and about which it has made either of the following findings, which shall be submitted to the chartering 


authority: 


(1) That the charter school has failed, or is unable, to implement the recommendations of the California 
Collaborative for Educational Excellence. 

(2) That the inadequate performance of the charter school, based upon an evaluation rubric adopted 
pursuant to Section 52064.5, is either so persistent or so acute as to require revocation of the charter. 

(c) The chartering authority shall consider increases in pupil academic achievement for all pupil 
subgroups served by the charter school as the most important factor in determining whether to revoke 


the charter. 


(d) A chartering authority shall comply with the hearing process described in subdivision (e) of Section 
47607 in revoking a charter. A charter school may not appeal a revocation of a charter made pursuant to 
this section. 


16 | P a g e 


Draft 8/24/17 




























